Skip to content

[turbo-tasks]: Manually devirtualize our internal calling conventions#93910

Draft
lukesandberg wants to merge 19 commits into
devirtualize_backend_operationsfrom
devirtualize_by_hand_since_rustc_cannot_do_it
Draft

[turbo-tasks]: Manually devirtualize our internal calling conventions#93910
lukesandberg wants to merge 19 commits into
devirtualize_backend_operationsfrom
devirtualize_by_hand_since_rustc_cannot_do_it

Conversation

@lukesandberg
Copy link
Copy Markdown
Contributor

No description provided.

Copy link
Copy Markdown
Contributor Author

lukesandberg commented May 18, 2026

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 18, 2026

Failing CI jobs

Commit: ee7f1dc | About building and testing Next.js

@lukesandberg lukesandberg changed the title Phase 1a: create empty turbo-tasks-handle crate [turbo-tasks]: Manually devirtualize our internal calling conventions May 18, 2026
Copy link
Copy Markdown
Contributor

@vercel vercel Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Suggestions:

  1. Type mismatch in eviction.rs causes undefined behavior: raw pointer cast to ProdHandleConcrete (parameterized on ProdBackingStorage = Either<TurboBackingStorage, NoopBackingStorage>) but the actual Arc holds TurboTasks<TurboTasksBackend<TurboBackingStorage>> — a different monomorphization.
  1. Missing static_handle feature on turbo-tasks-backend dev-dependency causes test panics with unreachable!() in from_static_raw().

Fix on Vercel

@lukesandberg lukesandberg force-pushed the devirtualize_by_hand_since_rustc_cannot_do_it branch from 2edc638 to f02ec68 Compare May 19, 2026 18:37
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 19, 2026

Stats from current PR

✅ No significant changes detected

📊 All Metrics
📖 Metrics Glossary

Dev Server Metrics:

  • Listen = TCP port starts accepting connections
  • First Request = HTTP server returns successful response
  • Cold = Fresh build (no cache)
  • Warm = With cached build artifacts

Build Metrics:

  • Fresh = Clean build (no .next directory)
  • Cached = With existing .next directory

Change Thresholds:

  • Time: Changes < 50ms AND < 10%, OR < 2% are insignificant
  • Size: Changes < 1KB AND < 1% are insignificant
  • All other changes are flagged to catch regressions

⚡ Dev Server

Metric Canary PR Change Trend
Cold (Listen) 821ms 862ms █████
Cold (Ready in log) 820ms 820ms ███▇▇
Cold (First Request) 1.304s 1.262s ▆▄▆▄▃
Warm (Listen) 862ms 812ms █████
Warm (Ready in log) 823ms 818ms █▇█▇▇
Warm (First Request) 635ms 658ms ▇▇▆▆▃
📦 Dev Server (Webpack) (Legacy)

📦 Dev Server (Webpack)

Metric Canary PR Change Trend
Cold (Listen) 810ms 811ms █▁███
Cold (Ready in log) 788ms 789ms ▆▁█▇▆
Cold (First Request) 3.219s 3.214s ▂▁▇▅▄
Warm (Listen) 811ms 810ms █▁███
Warm (Ready in log) 789ms 792ms ▆▁█▇▆
Warm (First Request) 3.235s 3.256s ▂▁▆▄▂

⚡ Production Builds

Metric Canary PR Change Trend
Fresh Build 5.114s 5.117s ▅▃▇▄▅
Cached Build 5.137s 5.166s ▆▅▇▆▅
📦 Production Builds (Webpack) (Legacy)

📦 Production Builds (Webpack)

Metric Canary PR Change Trend
Fresh Build 24.087s 24.206s ▂▁█▂▁
Cached Build 24.088s 23.955s ▂▁▆▂▁
node_modules Size 506 MB 506 MB ▁▁▁▁▁
📦 Bundle Sizes

Bundle Sizes

⚡ Turbopack

Client

Main Bundles
Canary PR Change
0_fjg8_hi5unh.js gzip 154 B N/A -
04hm05ar7kldw.js gzip 5.73 kB N/A -
0bw8hgkrnyz52.js gzip 169 B N/A -
0cz1d0mv5g_q7.js gzip 39.4 kB 39.4 kB
0dvitrl5zg37g.js gzip 8.82 kB N/A -
0sf7ysou-72zd.js gzip 8.71 kB N/A -
0u1yrhy0v3ukc.js gzip 65.6 kB N/A -
0wctkh6poo3-9.js gzip 152 B N/A -
0z_6ypvxbuu_a.js gzip 159 B N/A -
14e-v33-v2gy6.js gzip 155 B N/A -
157abun3hwc_s.js gzip 10.3 kB N/A -
1djx0fm1yw1tl.js gzip 70.9 kB N/A -
1elt1qium-r2m.css gzip 115 B 115 B
1jj68jv9537mc.js gzip 13.8 kB N/A -
1jpaub6y8xlfr.js gzip 2.3 kB N/A -
1kzhctatld4-r.js gzip 157 B N/A -
1ot0mvscrc_uf.js gzip 233 B N/A -
1smvc2zk5-ca-.js gzip 156 B N/A -
2_m3xv2uq3sjc.js gzip 1.46 kB N/A -
203qqfwjufsmt.js gzip 157 B N/A -
24y34mwgrkqp4.js gzip 8.78 kB N/A -
29p1h87nr475x.js gzip 157 B N/A -
2aljlkge5h1k9.js gzip 50.4 kB N/A -
2c-fd4y1zozz8.js gzip 8.79 kB N/A -
2d7416h_xd36x.js gzip 8.71 kB N/A -
2g21ny1t2kw37.js gzip 7.61 kB N/A -
2lyuhit6rn8fy.js gzip 9.44 kB N/A -
2q0gr8wfr3jwl.js gzip 8.77 kB N/A -
2t9e75oz6r0zp.js gzip 8.76 kB N/A -
2uku_olcn15b7.js gzip 8.79 kB N/A -
30r8mm-46bdqy.js gzip 220 B 220 B
355tnzfgll-w6.js gzip 153 B N/A -
3c1jdxkzlb8oq.js gzip 12.9 kB N/A -
3inab2jybr4k9.js gzip 450 B N/A -
3j0r9ohz0z01-.js gzip 154 B N/A -
3jkm5tdjvaf_q.js gzip 13.1 kB N/A -
3mt67agm5wp40.js gzip 10.6 kB N/A -
3saabek4kohwi.js gzip 10 kB N/A -
3sean5tdvwzz_.js gzip 155 B N/A -
3zan-tufsponr.js gzip 160 B N/A -
4189xmby9yu1p.js gzip 13.6 kB N/A -
turbopack-0k..eo4m.js gzip 4.2 kB N/A -
turbopack-0k..-9gf.js gzip 4.2 kB N/A -
turbopack-0o..1q02.js gzip 4.18 kB N/A -
turbopack-0q..5c5o.js gzip 4.2 kB N/A -
turbopack-16..fb3b.js gzip 4.2 kB N/A -
turbopack-17..j1a-.js gzip 4.2 kB N/A -
turbopack-1k..aoyi.js gzip 4.2 kB N/A -
turbopack-1r..h6ay.js gzip 4.2 kB N/A -
turbopack-1v..7iy-.js gzip 4.2 kB N/A -
turbopack-2d..rwvq.js gzip 4.21 kB N/A -
turbopack-2m..2k_2.js gzip 4.2 kB N/A -
turbopack-2o.._suq.js gzip 4.2 kB N/A -
turbopack-35..-lvd.js gzip 4.2 kB N/A -
turbopack-44..q5pn.js gzip 4.2 kB N/A -
0_i7nqgx23st7.js gzip N/A 10 kB -
06puhytyxk31p.js gzip N/A 8.82 kB -
0ey1zw-mlwx2h.js gzip N/A 155 B -
0j42f9zonj0wd.js gzip N/A 13 kB -
0m34gln_kt4fg.js gzip N/A 5.73 kB -
0p4sjm1_vsyqc.js gzip N/A 157 B -
0v8bak4y0xxj2.js gzip N/A 169 B -
0vwkqhru5pnu1.js gzip N/A 156 B -
10yxp2katn2c1.js gzip N/A 156 B -
1ck72gbqbfell.js gzip N/A 50.4 kB -
1g3q1ww01thnl.js gzip N/A 2.3 kB -
1hraqxuiymq6v.js gzip N/A 8.79 kB -
1l9un1sl77287.js gzip N/A 1.46 kB -
1ydp85rsgb8o6.js gzip N/A 155 B -
21-eavqb1k_36.js gzip N/A 13.9 kB -
2147zgtf14z-q.js gzip N/A 234 B -
234uq0f6lmp4e.js gzip N/A 70.9 kB -
23bz3xsg-5-1s.js gzip N/A 8.71 kB -
27441mytv7pbm.js gzip N/A 9.43 kB -
2cjkwjgm1zcfs.js gzip N/A 8.71 kB -
2mhqz28ez110r.js gzip N/A 158 B -
2qs_wqvnpva_z.js gzip N/A 155 B -
2ry6ynqi-5ptz.js gzip N/A 152 B -
2scd8zaoyb8md.js gzip N/A 8.79 kB -
2st_qs6p_9us0.js gzip N/A 13.1 kB -
2zo2exm1d8qj1.js gzip N/A 13.6 kB -
32kwi5zoqnqhy.js gzip N/A 161 B -
36-o575nl6lr_.js gzip N/A 156 B -
3a2xhwp6l10fw.js gzip N/A 156 B -
3f710q6kll2xn.js gzip N/A 7.61 kB -
3h7n9ebdmu7d3.js gzip N/A 159 B -
3hn75zuxly9az.js gzip N/A 10.3 kB -
3hqh7m128tvsn.js gzip N/A 8.77 kB -
3hqti_t-zy1x4.js gzip N/A 449 B -
3mnawenie1flm.js gzip N/A 8.76 kB -
3ubsozlu6zs38.js gzip N/A 10.6 kB -
42qezf5_oyvsu.js gzip N/A 65.6 kB -
43iwfqjnx1cy_.js gzip N/A 8.78 kB -
turbopack-02..61i-.js gzip N/A 4.2 kB -
turbopack-03..znl6.js gzip N/A 4.2 kB -
turbopack-0i..i6-4.js gzip N/A 4.2 kB -
turbopack-0p..68pi.js gzip N/A 4.2 kB -
turbopack-0x..aj2m.js gzip N/A 4.2 kB -
turbopack-11..azqn.js gzip N/A 4.2 kB -
turbopack-1p..vrdm.js gzip N/A 4.2 kB -
turbopack-1x..g7k6.js gzip N/A 4.2 kB -
turbopack-2b.._9pw.js gzip N/A 4.2 kB -
turbopack-2j..3lug.js gzip N/A 4.2 kB -
turbopack-2v..va8i.js gzip N/A 4.2 kB -
turbopack-3f..jnwg.js gzip N/A 4.21 kB -
turbopack-3j..0r1z.js gzip N/A 4.18 kB -
turbopack-3s..utxm.js gzip N/A 4.2 kB -
Total 469 kB 469 kB ⚠️ +90 B

Server

Middleware
Canary PR Change
middleware-b..fest.js gzip 716 B 717 B
Total 716 B 717 B ⚠️ +1 B
Build Details
Build Manifests
Canary PR Change
_buildManifest.js gzip 434 B 433 B
Total 434 B 433 B ✅ -1 B

📦 Webpack

Client

Main Bundles
Canary PR Change
2258-HASH.js gzip 61.4 kB N/A -
2266-HASH.js gzip 4.69 kB N/A -
3317.HASH.js gzip 169 B N/A -
4866-HASH.js gzip 5.64 kB N/A -
9e302639-HASH.js gzip 62.8 kB N/A -
framework-HASH.js gzip 59.5 kB 59.5 kB
main-app-HASH.js gzip 254 B 253 B
main-HASH.js gzip 39.9 kB 39.9 kB
webpack-HASH.js gzip 1.68 kB 1.68 kB
175fd0fd-HASH.js gzip N/A 62.8 kB -
2596-HASH.js gzip N/A 5.63 kB -
34-HASH.js gzip N/A 61.3 kB -
5691.HASH.js gzip N/A 169 B -
9156-HASH.js gzip N/A 4.68 kB -
Total 236 kB 236 kB ✅ -102 B
Polyfills
Canary PR Change
polyfills-HASH.js gzip 39.4 kB 39.4 kB
Total 39.4 kB 39.4 kB
Pages
Canary PR Change
_app-HASH.js gzip 193 B 193 B
_error-HASH.js gzip 181 B 182 B
css-HASH.js gzip 334 B 332 B
dynamic-HASH.js gzip 1.79 kB 1.81 kB
edge-ssr-HASH.js gzip 255 B 255 B
head-HASH.js gzip 351 B 348 B
hooks-HASH.js gzip 385 B 384 B
image-HASH.js gzip 580 B 580 B
index-HASH.js gzip 257 B 259 B
link-HASH.js gzip 2.51 kB 2.52 kB
routerDirect..HASH.js gzip 318 B 319 B
script-HASH.js gzip 387 B 386 B
withRouter-HASH.js gzip 316 B 316 B
1afbb74e6ecf..834.css gzip 106 B 106 B
Total 7.97 kB 7.99 kB ⚠️ +19 B

Server

Edge SSR
Canary PR Change
edge-ssr.js gzip 126 kB 126 kB
page.js gzip 276 kB 270 kB 🟢 5.28 kB (-2%)
Total 402 kB 396 kB ✅ -5.49 kB
Middleware
Canary PR Change
middleware-b..fest.js gzip 620 B 615 B
middleware-r..fest.js gzip 155 B 155 B
middleware.js gzip 44.4 kB 44.5 kB
edge-runtime..pack.js gzip 842 B 842 B
Total 46 kB 46.1 kB ⚠️ +80 B
Build Details
Build Manifests
Canary PR Change
_buildManifest.js gzip 719 B 717 B
Total 719 B 717 B ✅ -2 B
Build Cache
Canary PR Change
0.pack gzip 4.49 MB 4.49 MB
index.pack gzip 116 kB 115 kB
index.pack.old gzip 115 kB 113 kB 🟢 2.48 kB (-2%)
Total 4.72 MB 4.71 MB ✅ -7.3 kB

🔄 Shared (bundler-independent)

Runtimes
Canary PR Change
app-page-exp...dev.js gzip 350 kB 351 kB
app-page-exp..prod.js gzip 195 kB 195 kB
app-page-tur...dev.js gzip 350 kB 350 kB
app-page-tur..prod.js gzip 194 kB 194 kB
app-page-tur...dev.js gzip 346 kB 347 kB
app-page-tur..prod.js gzip 192 kB 192 kB
app-page.run...dev.js gzip 347 kB 347 kB
app-page.run..prod.js gzip 193 kB 193 kB
app-route-ex...dev.js gzip 77.5 kB 77.5 kB
app-route-ex..prod.js gzip 52.9 kB 52.9 kB
app-route-tu...dev.js gzip 77.6 kB 77.6 kB
app-route-tu..prod.js gzip 52.9 kB 52.9 kB
app-route-tu...dev.js gzip 77.2 kB 77.2 kB
app-route-tu..prod.js gzip 52.7 kB 52.7 kB
app-route.ru...dev.js gzip 77.1 kB 77.1 kB
app-route.ru..prod.js gzip 52.7 kB 52.7 kB
dist_client_...dev.js gzip 324 B 324 B
dist_client_...dev.js gzip 326 B 326 B
dist_client_...dev.js gzip 318 B 318 B
dist_client_...dev.js gzip 317 B 317 B
pages-api-tu...dev.js gzip 44.3 kB 44.3 kB
pages-api-tu..prod.js gzip 33.8 kB 33.8 kB
pages-api.ru...dev.js gzip 44.3 kB 44.3 kB
pages-api.ru..prod.js gzip 33.7 kB 33.7 kB
pages-turbo....dev.js gzip 53.7 kB 53.7 kB
pages-turbo...prod.js gzip 39.4 kB 39.4 kB
pages.runtim...dev.js gzip 53.6 kB 53.6 kB
pages.runtim..prod.js gzip 39.4 kB 39.4 kB
server.runti..prod.js gzip 63.1 kB 63.1 kB
use-cache-pr...dev.js gzip 69.7 kB 69.7 kB
use-cache-pr...dev.js gzip 69.7 kB 69.7 kB
use-cache-pr...dev.js gzip 68 kB 68 kB
use-cache-pr...dev.js gzip 68 kB 68 kB
Total 3.37 MB 3.37 MB ⚠️ +345 B
📝 Changed Files (17 files)

Files with changes:

  • app-page-exp..ntime.dev.js
  • app-page-exp..time.prod.js
  • app-page-tur..ntime.dev.js
  • app-page-tur..time.prod.js
  • app-page-tur..ntime.dev.js
  • app-page-tur..time.prod.js
  • app-page.runtime.dev.js
  • app-page.runtime.prod.js
  • pages-api-tu..ntime.dev.js
  • pages-api-tu..time.prod.js
  • pages-turbo...ntime.dev.js
  • pages-turbo...time.prod.js
  • server.runtime.prod.js
  • use-cache-pr..ntime.dev.js
  • use-cache-pr..ntime.dev.js
  • use-cache-pr..ntime.dev.js
  • use-cache-pr..ntime.dev.js
View diffs
app-page-exp..ntime.dev.js
failed to diff
app-page-exp..time.prod.js

Diff too large to display

app-page-tur..ntime.dev.js
failed to diff
app-page-tur..time.prod.js

Diff too large to display

app-page-tur..ntime.dev.js
failed to diff
app-page-tur..time.prod.js

Diff too large to display

app-page.runtime.dev.js
failed to diff
app-page.runtime.prod.js

Diff too large to display

pages-api-tu..ntime.dev.js

Diff too large to display

pages-api-tu..time.prod.js

Diff too large to display

pages-turbo...ntime.dev.js

Diff too large to display

pages-turbo...time.prod.js

Diff too large to display

server.runtime.prod.js

Diff too large to display

use-cache-pr..ntime.dev.js

Diff too large to display

use-cache-pr..ntime.dev.js

Diff too large to display

use-cache-pr..ntime.dev.js

Diff too large to display

use-cache-pr..ntime.dev.js

Diff too large to display

📎 Tarball URL
https://vercel-packages.vercel.app/next/commits/ee7f1dc65a915845cce83d48e62a7103d8ce5f3c/next

Commit: ee7f1dc

@lukesandberg lukesandberg changed the base branch from canary to graphite-base/93910 May 20, 2026 04:50
@lukesandberg lukesandberg force-pushed the devirtualize_by_hand_since_rustc_cannot_do_it branch from 7fffcd9 to 02d4b42 Compare May 20, 2026 04:50
@lukesandberg lukesandberg changed the base branch from graphite-base/93910 to 05-19-delete_vcstorage May 20, 2026 04:51
@lukesandberg lukesandberg changed the base branch from 05-19-delete_vcstorage to graphite-base/93910 May 20, 2026 17:40
@lukesandberg lukesandberg force-pushed the devirtualize_by_hand_since_rustc_cannot_do_it branch from 81ddf9c to f84c52e Compare May 20, 2026 17:40
@lukesandberg lukesandberg changed the base branch from graphite-base/93910 to simplify_no_op_backing_store May 20, 2026 17:40
This crate is the future home of the #[no_mangle] pub extern "Rust"
fn providers that implement the dispatch surface declared (later in
this phase) in turbo-tasks/src/handle.rs.

For now it is a skeleton: lib.rs is empty (just a doc comment) and
the crate exposes two optional features:
  - prod : pulls in turbo-tasks-backend (will emit __tt_prod_*
    providers).
  - test : pulls in turbo-tasks-testing (will emit __tt_test_*
    providers).

Building with neither feature compiles to an empty lib, so 'cargo
check --workspace' works without flags. Binaries and tests that
actually consume the dispatch must enable the appropriate feature(s).
Adds the tagged-pointer dispatch infrastructure that replaces the
Arc<dyn TurboTasksApi> task-local. Only one method (`invalidate`) is on
the dispatch surface so far — subsequent commits will add the rest.

In turbo-tasks/src/handle.rs:
  * TurboTasksHandle { tag: HandleTag, ptr: NonNull<()> } — the tagged
    pointer that will replace Arc<dyn TurboTasksApi> in TURBO_TASKS.
  * unsafe extern "Rust" forward declarations for __tt_prod_invalidate
    / __tt_test_invalidate.
  * impl TurboTasksHandle { fn invalidate(...) { match self.tag { ... } } }
    — direct dispatch over the tag; both arms are extern "Rust" calls
    that thin LTO will inline cross-crate.
  * Clone / Drop dispatch through __tt_<arm>_clone_arc / drop_arc so the
    underlying Arc refcount stays correct.
  * Uses macro_metavar_expr_concat (nightly, stable on our toolchain) to
    auto-generate the per-arm symbol names from a single method list.

In turbo-tasks-handle/src/lib.rs:
  * ProdHandleConcrete type alias matches what next-napi-bindings uses.
  * #[no_mangle] pub extern "Rust" fn __tt_prod_invalidate + clone/drop
    providers for the prod arm; mirror for the test arm (VcStorage).
  * from_prod / from_test constructors that convert an Arc<...> into
    a TurboTasksHandle by Arc::into_raw.

No call sites are switched yet; TURBO_TASKS still holds Arc<dyn
TurboTasksApi>. CI green for all feature combinations of
turbo-tasks-handle (none, prod, test, prod+test).
…d list

Adds the remaining ~21 methods of the TurboTasksApi + TurboTasksCallApi
trait surface to the tagged-pointer dispatch:

  TurboTasksCallApi:
    dynamic_call, native_call, trait_call, send_compilation_event,
    get_task_name

  TurboTasksApi:
    invalidate (already wired), invalidate_with_reason,
    invalidate_serialization, try_read_task_output, try_read_task_cell,
    try_read_local_output, read_task_collectibles, emit_collectible,
    unemit_collectible, unemit_collectibles, try_read_own_task_cell,
    read_own_task_cell, update_own_task_cell, mark_own_task_as_finished,
    connect_task, spawn_detached_for_testing,
    subscribe_to_compilation_events, is_tracking_dependencies

The following remain OFF the dispatch surface, per the plan:
  run, run_once, run_once_with_reason, start_once_process,
  stop_and_wait, task_statistics

Every caller of these already has a concrete TurboTasks<B> in scope
(via direct construction in #[tokio::test] / benches / fuzz / the napi
top-level run), so they don't need extern "Rust" dispatch.

Also switches the provider macro from UFCS `<Concrete as
TurboTasksApi>::` to method-call syntax `tt.`, which
resolves both supertrait (TurboTasksCallApi) and subtrait
(TurboTasksApi) methods uniformly.

Cleans up Arc clone/drop to use Arc::increment_strong_count /
decrement_strong_count instead of the from_raw -> clone -> forget
dance. Cleaner and avoids the sketchy debug_assert on pointer
identity.

All four feature combinations of turbo-tasks-handle compile:
none, prod, test, prod+test. Workspace check passes.
The thread-local will soon hold a TurboTasksHandle instead of an
Arc<dyn TurboTasksApi>, so turbo_tasks_weak() needs a parallel
TurboTasksWeakHandle type — the existing Arc::downgrade(arc) path
breaks once the task-local stops holding an Arc.

Adds:
  * pub struct TurboTasksWeakHandle { tag, ptr } — mirrors
    TurboTasksHandle but holds a Weak<concrete>::into_raw() pointer.
  * TurboTasksHandle::downgrade(&self) -> TurboTasksWeakHandle.
  * TurboTasksWeakHandle::upgrade(&self) -> Option<TurboTasksHandle>.
  * Per-arm extern "Rust" symbols: __tt_<arm>_downgrade, _upgrade,
    _clone_weak, _drop_weak. Providers in turbo-tasks-handle round-
    trip through Weak::from_raw/Weak::into_raw and Arc::downgrade /
    Weak::upgrade.

The plan suggested deferring weak-handle work to a later phase if
benches showed scope-spawning didn't matter. After auditing the
weak-handle consumers (turbo-tasks-fs's filesystem watcher, four
sites — all real and load-bearing), it's cleaner to build the weak
handle now alongside the strong handle than to leave a half-Weak<dyn>
hybrid behind.

No call sites switched yet.
…TurboTasksHandle (WIP)

This is the main flip. Workspace 'cargo check' is green; building test
binaries hits a linkage issue (see end of message).

Core change:
  * task_local! { TURBO_TASKS: TurboTasksHandle } in manager.rs (was
    Arc<dyn TurboTasksApi>).
  * Public accessors turbo_tasks(), try_turbo_tasks(), with_turbo_tasks,
    turbo_tasks_weak(), turbo_tasks_scope, turbo_tasks_future_scope,
    with_turbo_tasks_for_testing, and the free run/run_once/
    run_once_with_reason helpers now return/take TurboTasksHandle.
  * TurboTasksWeakHandle parallels TurboTasksHandle for the rare weak
    case (used by the fs watcher).

Method dispatch surface expanded:
  * run, run_once, run_once_with_reason, start_once_process,
    stop_and_wait, task_statistics are now also on the dispatch surface
    — the test harness in turbo-tasks-testing type-erases its
    TestInstance.tt and calls turbo_tasks::run_once() on it, so 'off
    the dispatch surface' didn't hold for these methods after all.
  * task_statistics returns &TaskStatisticsApi; the extern provider
    returns *const TaskStatisticsApi and the handle wrapper re-binds
    the lifetime to &self.
  * provide_prod_trait! / provide_test_trait! variants force UFCS for
    methods whose inherent signature on TurboTasks<B> shadows the
    trait method's signature (the run/stop family).

Internal API signature changes:
  * Invalidator::invalidate, invalidate_with_reason now take
    &TurboTasksHandle (was &dyn TurboTasksApi).
  * read_task_output, read_local_output internal helpers take
    &TurboTasksHandle. wait_task_completion's own loop is inlined.
  * TurboTaskContextError.turbo_tasks field changed from
    Arc<dyn TurboTasksCallApi> to TurboTasksHandle.
  * Backend operation::Context::turbo_tasks() returns TurboTasksHandle
    (uses thread-local).

Construction sites:
  * TurboTasks::make_handle(self: Arc<Self>) -> TurboTasksHandle
    inherent method on TurboTasks<B> and on VcStorage.
  * make_handle_from_ref(&self) for ergonomic use inside TurboTasks's
    own methods that need to scope into themselves.

Downstream updates:
  * turbo-tasks-fs DiskFileSystemInner.turbo_tasks Weak<dyn...> →
    TurboTasksWeakHandle.
  * turbo-tasks-fs watcher signatures changed.
  * turbopack-dev-server, turbopack-cli, turbopack-cli's serve()
    signature take TurboTasksHandle; UpdateServer::run too.
  * turbo-tasks-testing TestInstance.tt and helpers use handles.

Cargo deps:
  * next-napi-bindings dev-deps turbo-tasks-handle [features = 'prod'].
  * turbo-tasks-backend dev-deps turbo-tasks-handle [features =
    'prod', 'test'] for tests + benches.

** KNOWN ISSUE — TEST BUILD LINKAGE **

Test binaries fail to link because cargo doesn't pull
'libturbo_tasks_handle.rlib' into the test binary's link command even
though it's declared as a dev-dep. Investigation needed:
- Likely cause: cargo skips a dev-dep that's never referenced by name
  in the test sources.
- Quick workaround: add 'extern crate turbo_tasks_handle;' to each
  test file that lives in turbo-tasks-backend/tests/. ~30 files.
- Better solution: probably move providers to a separate
  turbo-tasks-prod-handle / turbo-tasks-test-handle pair of crates
  that consumers reference explicitly, or use a build-script trick
  to force linkage.

The cdylib (next-napi-bindings) does link correctly because cdylib
linking includes all declared deps.
…sting

The turbo-tasks-handle crate added in phase 1 created an awkward
cycle: it needed turbo-tasks-backend (for ProdHandleConcrete) and
turbo-tasks-testing (for VcStorage) as feature-gated deps, which made
adding it as a dep of turbo-tasks-backend or turbo-tasks-testing
impossible (cycle). The result was per-binary opt-in via dev-deps,
which doesn't compose: every test/bench/example crate would need to
explicitly enable features on a separate handle crate.

This commit collapses turbo-tasks-handle out of existence:

* The __tt_prod_* providers move into turbo-tasks-backend/src/
  handle_providers.rs. turbo-tasks-backend already owns
  TurboTasksBackend and the prod handle concrete type.
* The __tt_test_* providers move into turbo-tasks-testing/src/
  handle_providers.rs. turbo-tasks-testing already owns VcStorage.
* The dispatch contract (TurboTasksHandle, HandleTag, forward decls,
  forwarder macros) stays in turbo-tasks/src/handle.rs.
* Two new Cargo features on turbo-tasks: prod_handle, test_handle.
  Each gates its arm's HandleTag variant, extern decls, and match
  arms. Single-variant builds (only prod or only test) collapse to a
  direct call; both-variant builds get cmp + b.ne + direct call.
* turbo-tasks-backend deps on turbo-tasks [prod_handle].
  turbo-tasks-testing deps on turbo-tasks [test_handle].
* TurboTasks::make_handle is gated behind feature = 'prod_handle';
  VcStorage::make_handle is in turbo-tasks-testing so unconditional.

Test binaries linking the providers still has a wrinkle: when a
crate's test binary unifies test_handle on via dev-deps but the
test code doesn't 'use turbo_tasks_testing', cargo doesn't pull
libturbo_tasks_testing.rlib into the link command. The fix is
'extern crate turbo_tasks_testing;' in the affected files.

So far applied to:
- turbo-tasks-backend/src/lib.rs (#[cfg(test)])
- turbo-tasks-backend/tests/eviction.rs
- turbo-tasks-backend/benches/mod.rs

Still pending: a handful of other workspace crates that build tests
without using turbo_tasks_testing directly. Diagnosed but not yet
fixed (turbopack-cli, turbopack-tracing, etc.).
The retry/retry_async functions are generic Future utilities with no
dependency on turbo-tasks. They previously lived in
turbo-tasks-testing where they pulled the rest of that crate (and
its test_handle feature activation) into anything that transitively
needed retry.

Specifically, turbopack-cli dev-depped turbopack-bench which depped
turbo-tasks-testing solely for retry(). That dependency triggered
turbo-tasks feature unification that enabled test_handle, generating
extern decls for __tt_test_* in turbopack-cli's lib test binary,
which then failed to link because cargo doesn't pull the unused
turbo-tasks-testing rlib into the binary's link command.

Moving the two retry helpers to turbopack-bench (their sole user)
breaks that chain. turbopack-cli's lib tests now build cleanly.
When 'turbo-tasks' is built standalone with neither 'prod_handle' nor
'test_handle' active, 'HandleTag' previously became a zero-variant
enum which is invalid under '#[repr(u8)]' — and downstream consumers
(e.g. 'turbo-tasks-bytes') that only declare value types would fail
to compile because TurboTasks::make_handle requires
'HandleTag::Prod'.

This commit:
- Adds 'HandleTag::Unreachable = 255', present only when neither
  feature is active. The variant is never constructed by consumer
  code; it just keeps the enum inhabited.
- Adds '_ => unreachable!()' arms to every dispatch 'match'
  (TurboTasksHandle's forwarder macro, task_statistics, Clone,
  Drop, downgrade, and the same trio for TurboTasksWeakHandle),
  feature-gated to only exist when no real arm exists.
- Ungates TurboTasks::make_handle / make_handle_from_ref so they
  always exist; in the no-features build they construct a handle
  with the Unreachable tag (any dispatch through it panics at
  runtime, but the binary builds).
- Adds 'compile_error!' if a build activates 'test_handle' but not
  'prod_handle' AND tries to call TurboTasks::make_handle — that
  combination is structurally inconsistent.

The workspace 'cargo check' is green. 'cargo build --workspace
--tests --benches' still has linker errors in crates that don't
directly dep on turbo-tasks-backend or -testing but get their
features unified on by other workspace members. Fixing those needs
'extern crate' anchors in the affected crates (next commit).
Renames the dispatch arms by mechanism (Static/Dynamic) rather than by
intended user (Prod/Test). VcStorage from turbo-tasks-testing uses the
Dynamic arm because we don't want to wire its concrete type into the
static dispatch surface, not because dynamic dispatch is inherently a
"test" concept.

Also restructures features so static_handle is explicit opt-in
(activated by turbo-tasks-backend's static_handle feature, which the
napi binding enables) rather than being feature-unified workspace-wide.
This avoids the linker error where test binaries that don't directly
link turbo-tasks-backend would otherwise see unresolved __tt_static_*
extern symbols.

Drops the unused read_task_output free function.
The `__tt_static_*` providers in turbo-tasks-backend cast the opaque
`*const ()` receiver to `&TurboTasks<TurboTasksBackend<Either<TurboBackingStorage,
NoopBackingStorage>>>`. Tests and benches were constructing
`TurboTasks::new(TurboTasksBackend::new(_, TurboBackingStorage))` (no
Either wrapper), so the resulting `Arc<TurboTasks<_>>` had a different
concrete type than what the providers expected — reinterpreting the
pointer was undefined behavior and segfaulted under optimization.

Adds `ProdBackingStorage` (the Either-wrapped type) and small
`prod_backing_storage_turbo` / `prod_backing_storage_noop` helpers in
turbo-tasks-backend, then updates every test_config.trs and the three
bench harnesses to use them.

Also gates the backend's own tests/benches to activate the
`static_handle` feature on themselves via a dev-dep self-reference, so
`TurboTasksHandle::from_static_raw` is wired up to the actual extern
providers instead of the no-feature unreachable! stub.
…nd self-dev-dep

`turbo_tasks()` returns a `TurboTasksHandle` by value, not a smart pointer
that can be deref'd. The new caller in client.rs needs `&turbo_tasks()`
not `&*turbo_tasks()` — bind to a local first so the temporary's drop
order is explicit.

Also reverts the self-dev-dep on turbo-tasks-backend that activated
`static_handle` for its own tests/benches. That activation propagated
across the workspace via Cargo feature unification and broke test bins
that don't directly link `turbo-tasks-backend` (unresolved
`__tt_static_*` extern symbols). Backend tests now panic at runtime
without `static_handle`; will reactivate via per-crate dev-deps in a
follow-up so only test bins that actually link the backend turn it on.
Resolves the bench-job panic without forcing `static_handle` on
workspace-wide (which would spread `__tt_static_*` extern decls into
test bins that don't link `turbo-tasks-backend`).

When `static_handle` is on (napi binding), `make_handle` returns the
Static handle that dispatches through `extern "Rust"` symbols —
devirtualized under thin LTO, as before.

When `static_handle` is off but `dynamic_handle` is on (workspace
tests/benches, which transitively pull in `turbo-tasks-testing`),
`make_handle` upcasts the `Arc<TurboTasks<B>>` to `Arc<dyn
TurboTasksApi>` and returns a Dynamic handle. One vtable indirection
per dispatched call — slower than static, but correct, and only the
test/bench path pays it.

The third arm (neither feature on) is a runtime panic stub so the
generic `impl<B> TurboTasks<B>` block still compiles for crates that
only depend on `turbo-tasks` for types.
Removes both the `static_handle` (on `turbo-tasks`) and `dynamic_handle`
(now dead since VcStorage was deleted) Cargo features.
`TurboTasksHandle` is now a single-variant `NonNull<()>`-wrapping type
that always dispatches through `extern "Rust"` symbols defined in
`turbo-tasks-backend`.

This means every binary that links `libturbo_tasks.rlib` must also link
`libturbo_tasks_backend.rlib`. Most workspace crates that depend on
`turbo-tasks` for runtime use already do. For type-declaring leaf
crates (`turbo-tasks-bytes`, `turbo-esregex`, `turbopack-css`, etc.)
whose lib-test binaries link the rlib but otherwise have no need for
the backend, the convention is a `[dev-dependencies] turbo-tasks-backend
= { workspace = true }` plus `#[cfg(test)] extern crate
turbo_tasks_backend;` in `src/lib.rs` — cargo won't include the rlib in
the link command unless Rust code references it.

A follow-up could replace the unconditional extern decls with weak
symbols (`#[linkage = "extern_weak"]`), letting the dispatch site
resolve to a null function pointer for crates that never actually
construct a `TurboTasks<B>`. Under thin LTO the resulting null-check
should constant-fold away in callers that do link the backend. Not
done here because it would require nightly-only `linkage` attribute
support and `Option<unsafe extern "Rust" fn(...)>` plumbing.
Each dispatched method previously required two macro invocations that duplicated
the signature: tt_decl_extern! for the extern declaration and tt_decl_handle_method!
for the matching inherent method on TurboTasksHandle. Merge them into a single
tt_decl! that emits both.
The only caller (next-napi-bindings/src/next_api/project.rs:2142) holds the
concrete `Arc<TurboTasks<TurboTasksBackend<TurboBackingStorage>>>` and goes
through the trait method via static dispatch. No call site reaches it via
`TurboTasksHandle`, so the extern + handle wrapper were dead weight.
Both APIs previously took a `TurboTasksHandle`, forcing callers like
turbopack-cli to erase their concrete `Arc<TurboTasks<Backend>>` through
the extern-dispatch handle. Make them generic over `T: TurboTasksApi`
and `T: TurboTasksCallApi` respectively, so the concrete backend type
flows through and trait methods monomorphize directly.

The two `run_once_with_reason` free-helper calls in dev-server are
inlined: the request path uses a oneshot channel to ferry the response
out of the `Result<()>` trait method, and the side-effects path spawns
its closure inside `tokio::spawn(async move { tt.run_once_with_reason(...) })`
so the boxed future owns its `Arc<T>` clone (the trait method's future
isn't `'static`).
…andle

These four methods were only routed through the type-erased
TurboTasksHandle so test code could hold them. Every production caller
already has the concrete Arc<TurboTasks<TurboTasksBackend<_>>> and uses
the inherent methods directly.

Make TestInstance carry the concrete arc
(Arc<TurboTasks<TurboTasksBackend<TurboBackingStorage>>>) instead of a
TurboTasksHandle — all .trs files produce that exact type, so
Registration / TestInstance stay non-generic. Migrate the test-only
turbo_tasks::run / run_once / run_once_with_reason free-fn shims into
turbo-tasks-testing, where they can call the inherent methods (which
already return Result<T> directly, so the oneshot-channel wrapper is
unnecessary).

Then delete: the four tt_decl entries, their providers in
handle_providers.rs, the now-unused provide_prod_trait! macro, the
three free-fn shims in manager.rs, and the matching trait methods on
TurboTasksCallApi (run, run_once) and TurboTasksApi (stop_and_wait).
TurboTasksCallApi::run_once_with_reason stays — the dev-server uses it
through Arc<dyn TurboTasksApi>.
@lukesandberg lukesandberg changed the base branch from simplify_no_op_backing_store to graphite-base/93910 May 21, 2026 01:15
@lukesandberg lukesandberg force-pushed the devirtualize_by_hand_since_rustc_cannot_do_it branch from d4dde43 to ee7f1dc Compare May 21, 2026 01:15
@lukesandberg lukesandberg changed the base branch from graphite-base/93910 to devirtualize_backend_operations May 21, 2026 01:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant